feat(core): Add fieldSelector to events_list and namespaces_list tools#1138
Conversation
Signed-off-by: Jacek Luczak <difrost.kernel@gmail.com>
Signed-off-by: Jacek Luczak <difrost.kernel@gmail.com>
|
Consider adding a unit test for |
…Selector Signed-off-by: Jacek Luczak <difrost.kernel@gmail.com>
| @@ -0,0 +1,2 @@ | |||
| #!/usr/bin/env bash | |||
| kubectl delete namespace event-filter-test --ignore-not-found | |||
There was a problem hiding this comment.
should it be events-filter-test ?
Signed-off-by: Jacek Luczak <difrost.kernel@gmail.com>
There was a problem hiding this comment.
Thanks for tackling this. events_list on a busy namespace really was a footgun, and the implementation cleanly mirrors pods_list. The signature change to EventsList(ctx, namespace, options) is consistent with NamespacesList and PodsListInAllNamespaces, all call sites are updated (the kubevirt vm_troubleshoot.go migration is correct), and the snapshot updates cover core, full, openshift and multicluster.
A couple of things I'd like addressed before merge. Both are scoped tightly enough to fit this PR, and one is a missed opportunity that the new signature begs for.
Must fix
1. REGEX_FIELDSELECTOR (pkg/toolsets/core/toolset.go:14) rejects values the tool says it supports.
The events_list description lists involvedObject.apiVersion as a supported selector field. Realistic values for that include apps/v1, batch/v1, events.k8s.io/v1, all of which contain /, which the pattern ^[.\-A-Za-z0-9]+([=!,]{1,2}[.\-A-Za-z0-9]+)+$ does not allow. Verified empirically:
"involvedObject.apiVersion=apps/v1" => false
So a spec-compliant MCP client will refuse the call before it reaches the apiserver. Adding / to the character class is safe for the other tools that share this regex. Pods, Namespaces and Resources selector values are DNS-1123 and never contain /, so widening cannot make a previously-invalid selector match.
Side note: the same regex also accepts garbage like a==b, a,b, a!=b!=c. That's a pre-existing wart, fine to leave for a follow-up, but please add at least one positive test for a comma-separated multi-clause selector while you're in regex_test.go.
2. projects_list should gain the same parameter (pkg/toolsets/core/namespaces.go).
Withdrawn. @difrost was right: the OpenShift Project
Listendpoint silently drops the field selector (proxy.go L95-L106), so addingfieldSelectortoprojects_listwould be a no-op. See the thread for the full receipt.
OpenShift projects_list supports the same field selectors as Namespaces (metadata.name, status.phase). Leaving it out creates an asymmetry where the same agent gets the filtering capability on Kubernetes but loses it on OpenShift. Easy mirror of namespacesList.
3. vm_troubleshoot.go should push the filter server-side (pkg/toolsets/kubevirt/vm_troubleshoot.go:361).
fetchEvents currently lists every event in the namespace and then filters client-side by involvedObject.name matching the VM (or virt-launcher-<vm> prefix). With your new option, the exact-name match can be pushed to the apiserver, which is exactly the kind of busy-namespace cost this PR is meant to remove. A simple form: FieldSelector: "involvedObject.name=" + vmName, while keeping the existing client-side prefix check for virt-launcher-* pods (prefix match isn't expressible as a fieldSelector).
Optional but nice to have
4. Eval verifications don't prove the model used fieldSelector.
contains: "bad-pod" and contains: "ns-beta" both pass for unfiltered listings, so today these evals can't distinguish "agent learned the new parameter" from "agent listed everything". I see you flagged this in the description, and the framework's grammar (contains, llmJudge.contains, verify.sh) doesn't ship a native not_contains. The pragmatic option is a verify.sh that captures the response and greps for the negative case, or one that re-invokes the tool with the expected selector and compares the count. At minimum, a TODO in the task YAML noting the limitation would be useful.
5. Regex test coverage (pkg/toolsets/core/regex_test.go:36-51).
Once #1 is addressed, please add positive cases for:
type=Warning,reason=BackOff(comma-separated)involvedObject.apiVersion=apps/v1(the case that motivated the fix)
A Test_FieldSelectorRegex_is_invalid table with whitespace, empty and garbage cases would be valuable too but I'm fine deferring it.
Assessment
| "fieldSelector": { | ||
| Type: "string", | ||
| Description: "Optional Kubernetes field selector to filter events by field values (e.g. 'type=Warning', 'involvedObject.name=my-pod'). Supported fields: involvedObject.kind, involvedObject.name, involvedObject.namespace, involvedObject.uid, involvedObject.apiVersion, involvedObject.resourceVersion, involvedObject.fieldPath, reason, reportingComponent, source, type. See https://kubernetes.io/docs/concepts/overview/working-with-objects/field-selectors/", | ||
| Pattern: REGEX_FIELDSELECTOR, |
There was a problem hiding this comment.
REGEX_FIELDSELECTOR (defined at pkg/toolsets/core/toolset.go:14) rejects involvedObject.apiVersion=apps/v1 because / isn't in the character class, yet the description right above advertises involvedObject.apiVersion as supported. Verified: "involvedObject.apiVersion=apps/v1" does not match the pattern.
Adding / to the character class is safe for the other callers (pods_list, namespaces_list, resources_list) since their advertised field values are DNS-1123 and never contain /.
While you're touching this, please also add a positive case for a comma-separated multi-clause selector to pkg/toolsets/core/regex_test.go.
There was a problem hiding this comment.
Implemented in 076c695 including the test for invalid pattern.
| "fieldSelector": { | ||
| Type: "string", | ||
| Description: "Optional Kubernetes field selector to filter namespaces by field values (e.g. 'metadata.name=default', 'status.phase=Active'). Supported fields: metadata.name, status.phase. See https://kubernetes.io/docs/concepts/overview/working-with-objects/field-selectors/", | ||
| Pattern: REGEX_FIELDSELECTOR, |
There was a problem hiding this comment.
OpenShift projects_list (just below, around line 41) supports the same field selectors as Namespaces (metadata.name, status.phase). Could you mirror this fieldSelector block onto projects_list so the OpenShift path doesn't lose the new filtering capability?
There was a problem hiding this comment.
@manusa when I've check on my OCPs I was not able to filter by e.g. metadata.name and assumed this doesn't work. Could be a bug or sth ... I was checking on 4.18.
There was a problem hiding this comment.
You're right, and sorry for sending you on a goose chase. I went and read the openshift-apiserver source: the Project List endpoint deliberately drops the field selector.
In pkg/project/apiserver/registry/project/proxy/proxy.go:
func (s *REST) List(ctx context.Context, options *metainternal.ListOptions) (runtime.Object, error) {
...
labelSelector, _ := apihelpers.InternalListOptionsToSelectors(options)
namespaceList, err := s.lister.List(user, labelSelector)
...
}apihelpers.InternalListOptionsToSelectors returns (labelSelector, fieldSelector) but the project proxy assigns the field selector to _ and only forwards the label selector to s.lister.List. So fieldSelector on oc get project is a silent no-op on the List path. Interestingly, Watch on the same file does honor both selectors via MatchProject, which is why projectToSelectableFields in pkg/project/util/util.go still declares metadata.name, metadata.namespace, status.phase — List just never reaches it.
So your empirical finding on 4.18 was correct and leaving projects_list alone is the right call. Withdrawing #2, thanks for pushing back.
There was a problem hiding this comment.
Yee, the API lists support for fieldSelector so indeed it's really confusing to find out that this feature is not actually implemented.
| func fetchEvents(ctx context.Context, client api.KubernetesClient, namespace, vmName string) string { | ||
| core := kubernetes.NewCore(client) | ||
| eventMap, err := core.EventsList(ctx, namespace) | ||
| eventMap, err := core.EventsList(ctx, namespace, api.ListOptions{}) |
There was a problem hiding this comment.
Now that EventsList accepts a FieldSelector, this is a great place to push the kind/name match server-side instead of listing every event in the namespace and filtering client-side below. Something like:
opts := api.ListOptions{ListOptions: metav1.ListOptions{FieldSelector: "involvedObject.name=" + vmName}}
eventMap, err := core.EventsList(ctx, namespace, opts)The virt-launcher-<vm> prefix branch still has to stay client-side (prefix match isn't expressible as a fieldSelector), but the exact-name match for VM and VMI events can move to the apiserver. That directly addresses the busy-namespace cost this PR is meant to fix.
| setup: | ||
| file: setup.sh | ||
| verify: | ||
| contains: "bad-pod" |
There was a problem hiding this comment.
contains: "bad-pod" passes for an unfiltered listing too (the Normal good-pod events plus the Warning bad-pod events both contain the string), so this eval can't distinguish a model that learned fieldSelector from one that didn't.
The framework doesn't ship a native not_contains, but a verify.sh that captures the response and asserts good-pod is absent would actually exercise the parameter. At minimum, please add a TODO noting this gap.
There was a problem hiding this comment.
I was looking for a way to set this eval up correctly and checked whether mcpchecker passes the response to the script in a given stage but I couldn't make it working (CC: @Cali0707). Is this feature really there? That would be very handy.
| setup: | ||
| file: setup.sh | ||
| verify: | ||
| contains: "ns-beta" |
There was a problem hiding this comment.
Same concern as the events eval: contains: "ns-beta" passes for an unfiltered namespaces_list (it would still return all three of ns-alpha, ns-beta, ns-gamma). Consider a verify.sh that asserts ns-alpha and ns-gamma are absent.
There was a problem hiding this comment.
Implemented and tested for both evals in f90ea7b.
…Selector options in regex tests; use filtering by involvedObject in kubevirt vm troubleshooting Signed-off-by: Jacek Luczak <difrost.kernel@gmail.com>
Signed-off-by: Jacek Luczak <difrost.kernel@gmail.com>
manusa
left a comment
There was a problem hiding this comment.
Thanks for the quick turnaround. All three must-fix points are addressed:
- #1 regex now accepts
apps/v1-style values (/added to the value-side character class only, conservative and correct). - #3
fetchEventspushes the exact-name match server-side. - #5 regex_test.go gained the comma-separated positive case I asked for, plus a new
Test_FieldSelectorRegex_is_invalidtable with whitespace/empty/garbage cases. Nicely above and beyond. - #4 evals migrated to
mcpchecker/v1alpha2and the verify steps now actually exercise the filter (events checksbad-podpresent ANDgood-podabsent; namespaces asserts no non-ns-betaoutput).
LGTM, approving.
This PR adds
fieldSelectorproperty toevents_listandnamespaces_listin a similar fashion how it's defined for e.g.pods_list.The biggest added values is for the
events_listwhere a tool call on a busy namespace with a lot of events generated is killing most of the agents.I've also added evals for both tool extensions but can't figure a better way than just verifying with llmJudge
contains.@Cali0707 / @manusa please review.